Submit New Data

Submit your existing dataset to quickly, and automatically, run it through a series of predefined quality control (QC Quality Control) checks and transform data schema into a standard data target. Once processed, GIS Geographic Information System. A computer application that involves storing and manipulating electronic maps and related data. Also, mapping software combining spatial information about where places and events are located with data attributes describing those places and events. professionals can use the results to easily identify where there are opportunities to improve data quality.

GIS Data Hub provides results based on the accuracy and integrity of your submitted dataset. Please note that the most recent version of any layer or table is kept by GIS Data Hub at any given time, regardless of how well it performed against quality control (QC) checks. If your data fails to meet the level of quality you expected, you can upload a different version of that particular layer or table to replace it for future jobs. GeoComm recommends uploading the most quality and accurate version of your data at all times.

When submitting your data, you have the option to submit a complete or partial dataset as defined below. Files should be submitted following the requirements and guidelines detailed here: Requirements and Guidelines for Submitting a Data File.

  • Complete Dataset: Includes ALL layers and tables your agency requires GIS Data Hub to inspect. See Submit a Complete Dataset.

  • Partial Dataset: Includes a SUBSET of layers and tables from the agency's complete dataset. See Submit a Partial Dataset.

    Important: To submit a partial dataset, a complete dataset must have been previously submitted.

 

After you have submitted your data, the Upload Activity pane updates to show the data is uploading and processing. See View Upload Activity.

There are certain cases where GIS Data Hub recognizes that more information is needed to complete the upload. When this happens, the current upload stops and an e-mail providing additional instructions is sent to the e-mail address that was configured to receive notifications. The following provides some scenarios when the above may occur:

  • Submit data files as a single zip file no larger than 5 GB.

  • Points, multipoints Multipoints are features that are composed of more than one point., lines, and polygon feature classes are supported.

  • FGDB and SHP's can be nested up to 10 levels deep within the zip file.

    • Nested folders may contain any naming convention.

    • Any data located over 10 levels deep is ignored.

    • Keep in mind that zipping your upload creates an additional level.

    • If preferred, nested folders may be zipped.

  • Submit data files as a feature class or table within a single or multiple FGDBs, or as a SHP. These files can include Z or M values.

  • Empty or null records can be included in your upload, however, these records are stripped once the data is transformed into the target database In software, a collection of data organized in a disciplined, logical structure so users can have quick access to it..

  • Submitted folders that end in .gdb must be geodatabases. Folders ending in .gdb that do not contain GDB artifacts are considered invalid and are skipped. To avoid issues, rename non-GDB's so they do not end with a .gdb extension.

  • Single or multiple Excel or CSV files are accepted inside the submitted zip file.

    • Acceptable file format types include: .xlsx, .xlsm, and .csv.

      .xls files are not supported.

    • Files can be placed inside the main zip file or nested files within the zip file.

    • Excel or CSV files are limited to 100 MB. Files over this limit can be submitted but are not fully supported and also will cause the job to slow down.

      For larger files, consider breaking into multiple files.

    • The Excel or CSV files can be included in the zip file with other GDH supported file types or alone.

    • When an Excel file is uploaded with multiple sheets, each sheet in the file is converted to an individual database table within GIS Data Hub and used in geodatabase outputs created.

    • When a CSV file is uploaded, the individual sheet is converted to a single, individual table. The table name is used as the file name.

      Important: Keep in mind that changing data formats nearly always causes schema changes. This is especially true whenever you change data formats such as switching from an .xlsx to a CSV, changing sheet names in Excel/CSV files, or updating column headers.

      When format changes exist, it is best to make all format changes at once, then notify GeoComm staff after the complete dataset is uploaded so proper configuration changes can be made.

    • When naming your Excel or CSV files, the sheet name and column names should be limited to 63 characters.

      • When the file is submitted with character lengths over 63, the column headers from the Excel or CSV files are truncated to the 63 character maximum.

      • Column headers and special characters:

        • Column headers are required. If a header is not included in your file(s), the first values in the Excel or CSV file are used as the header row.

        • Asterisks: Column headers from your Excel or CSV files may contain asterisks. The asterisks cannot be the final character in the header.

        • Asterisks: Column headers from your Excel or CSV files that exceed 63 characters, and ends in an asterisk, are not supported—they are ignored from the upload. All other headers are read and the job processes as expected.

        • Special characters in your column headers are converted to underscores.

    • Tables converted to Excel in ArcMap or ArcGIS Pro may have field name changes or additional fields added during the conversion (i.e., adding Object ID). If using ArcMap or ArcGIS Pro to convert to Excel, verify your field names are correct before submitting your data.

    • If processing of the file(s) fails, an e-mail is sent with failure information.

  • Multipoint features classes are accepted.

    • The object ID of multipoint feature classes are not honored due to each spatial point being "exploded" into one tabular record.

    • When there are identical multipoint features in both the spatial location (coordinate) and attribution, they become one record in the output when "exploded."

    • When swapping data formats (i.e., feature class to SHP, table to CSV), a complete dataset upload must be completed, and field mapping updated as necessary, before running a job.

      Note that data conversion generally create schema changes that could cause a partial upload to fail.

  • Various projections for spatial data types can be used, as long as they are not custom projections.

    It is recommended to submit data whenever possible in the projection of the sought after output. This provides greater control over potential spatial data shifts related to re-projections and transformations.

  • GIS Data Hub reads up to 500 total data files (feature classes/shapefiles/tables). These files can be nested within the .zip file.

  • Feature classes or Shapefiles may contain multiple OBJECTID or FID fields upon upload. Shapefiles with an OBJECTID field must contain non-zero values for that field.

  • Invalid file types are ignored and not ingested by GIS Data Hub. Examples of invalid file types include PowerPoint Presentations, PDF files, and Word documents.

  • Multiple (up to 25) FGDB's are supported within a single zip file.

  • Special Esri Environmental Systems Research Institute, Inc. A company that designs and develops GIS technology. data types are supported and can be included in your data upload. However, the data types cannot be used within GIS Data Hub or exported.

    • Raster Datasets: May be included with your data upload but it is not recommended due to the size of a raster dataset.

    • Data types that can be uploaded

      While the following data types may be included in your data upload and will not stop job processing, they cannot be configured for use in any QC check or field mapping.

      • Annotation Feature Classes

      • Attribute Rules

      • Catalog Datasets

      • Dimension Feature Classes

      • Geometric Networks

      • Locators

      • Mosaic Datasets

      • Network Datasets

      • Oriented Imagery

      • Parcel Fabrics

      • Relationship Classes

      • Topologies

      • Trace Networks

      • Trajectory Datasets

      • Utility Networks

    • Data types that must never be uploaded

      If the following data types are included in your data upload to GIS Data Hub, your data upload will fail.

      • 3D Object Feature Classes

      • Linear Referencing Classes (LRS)

      • Multipatch Feature Classes

      • Raster Catalogs

  • The following file extensions are required when submitting a SHP: .shp, .shx, .dbf, and .prj.

    Important: If changing the file format for the data being uploaded (i.e., uploading a SHP instead of a feature class for a layer that was previously uploaded), it is highly recommended that the upload file is in a complete dataset the first time it is uploaded.

    Uploading a complete dataset avoids errors where there have been truncations or name changes to a layer and/or field name as a result of the format conversion. For details on Shapefile limitations see Shapefiles in ArcGIS Pro documentation.